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SYSTEM AND METHOD FOR COGNITIVE 
PROCESSING FOR DATA FUSION 

CROSS-REFERENCE TO RELATED 

APPLICATIONS 5 

This application is related to and claims the benefit of the 
following copending and commonly assigned U.S. Provi- 
sional Patent Application: U.S. Patent Application No. 
61/314,055, titled “Real Time Cognitive Computing Archi- 10 
tecture for Data Fusion in a Dynamic Environment,” filed on 
Mar. 15, 2010; the entire contents of which is incorporated 
herein by reference. 

STATEMENT OF GOVERNMENT GRANT 15 

The invention described herein was made in the perfor- 
mance of work under a NASA contract, and is subject to the 
provisions of Public Law 96-517 (35 USC 202) in which the 
Contractor has elected to retain title. 20 

BACKGROUND 

1. Field 

This disclo sure relates to a system and method for real time 25 
cognitive processing for data fusion in a dynamic environ- 
ment. More particularly, the present disclosure describes a 
method and system for real-time, adaptive, intelligent, low 
power, high productive and miniaturized processing using 
custom VLSI design for target detection and classification. 30 

2. Description of Related Art 

A general purposed computer can typically simulate or 
support just about any application through iterated computa- 
tion in a sequential manner. Such general purpose computers 
typically use an architecture known as a Von Neumann archi- 35 
tecture that consists of input/output, memory, and an Arith- 
metic Logic Unit. The Von Neumann architecture (or Von 
Neumann machine) is a sequential computation architecture. 
One draw back of the Von Neumann architecture is that it is 
slow, regardless of computer speed. 40 

To deal with complex data fusion applications as required 
in military applications, particularly remote, real time appli- 
cations related to the dynamic environment, the Von Neu- 
mann machine may not be effective for demands such as 
compactness, real time processing, adaptive system and low 45 
power. There are many types of data (e.g., IR, LIDAR, 
RADAR, Visual, Olfactory) that need to be processed and 
fused in real time. In data processing and fusion, especially 
for sensors, time can be critical to every millisecond. The 
speed requirements may present a challenge for a digital 50 
computer and the architecture of a system as a whole. For 
example, sensors may collect analog data that will be con- 
verted to a digital format before sending the data to the digital 
machine in a sequential manner for algorithmic computation. 
Each sequential step requires a delay and processing time to 55 
digest data, and finally, the solution that is provided by the 
computer may no longer be valid. 

In contrast, a neural network architecture is a parallel pro- 
cessing technique and its performance can be much faster as 
compared with the digital machines. See, for example, J. J. 60 
Hopfield, “Neural networks and physical systems with emer- 
gent collective computational abilities,” Proc. Natl. Acad. 

Sci. USA, vol. 79, pp. 2554-2558, 1982 and T. A. Duong, S. 

P. Eberhardt, T. Daud, and A. Thakoor, “Learning in neural 
networks: VLSI implementation strategies,” Fuzzy logic and 65 
Neural Network Handbook, Chap. 27, Ed: C. H. Chen, 
McGraw-Hill, 1996. However, neural network hardware is 
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typically not as fully-programmable as a digital computer. In 
neural network, one computer (or set of computers) may 
perform the learning and download a weight set to a neural 
network chip (see for example, E. Fiesler, T. Duong, and A. 
Trunov, “Design of neural network-based microchip for color 
segmentation,” SPIE Proceeding of Applications and Science 
of Computational Intelligence part III, Vol. 4055, pp. 228- 
237, Florida, May 2000, and T. X. Brown, M. D. Tran, T. A. 
Duong, T. Daud, and A. P. Thakoor, Cascaded VLSI neural 
network chips: Hardware learning for pattern recognition and 
classification, Simulation, 58, 340-347, 1992), while another 
computer (or set of computers) can perform on-chip learning 
with limited programming capability (i.e., not flexible for 
other applications) (see, for example, T. A. Duong, T. Daud, 
“On-Chip Learning of Hyper-Spectra Data for Real-Time 
Target Recognition”, Proceeding of IASTED Intelligent Sys- 
tem and Control, pp. 305-308, Honolulu, Hi., August 14-17, 
2000; B. Girau, “On-Chip learning of FPGA-Inspired neural 
nets,” Proceeding of International of Neural Networks 
IJCNN’2001, Vol. 1, pp. 212-215, 2001; C. Lu, B. Shi, andL. 
Chen, “A Programmable on-chip learning neural network 
with enhance characteristics,” The 2001 IEEE Inter. Sympo- 
sium Circuit and Systems ISCAS 2001, Vol. 2, pp. 573-576, 
2001; and G. M. Bo, D. D. Caviglia, and M. Valle, “An 
on-chip learning neural network,” Proc. Inter. Neural Net- 
works IJCNN’2000, Vol. 4, pp. 66-71, 2000). A neural net- 
work hardware implementation also has a two -fold problem: 
reliable learning techniques in limited weight space for learn- 
ing network convergence in a parallel architecture (see, for 
example, T. A. Duong and Allen R. Stubberud, “Convergence 
Analysis Of Cascade Error Projection — An Efficient Learn- 
ing Algorithm For Hardware Implementation,” International 
Journal of Neural System, Vol. 10, No. 3, pp. 199-210, June 
2000; Hoehfeld, M. andFahlman, S., “Learning with limited 
numerical precision using the cascade-correlation algo- 
rithm,” IEEE Trans. Neural Networks, vol. 3, No. 4, pp. 
602-611, 1992; and P. W. Hollis, J. S. Harper, and J. J. Paulos, 
“The Effects of Precision Constraints in a Backpropagation 
learning Network,” Neural Computation, vol. 2, pp. 363-373, 
1990), and a flexible architecture to solve a wide range of 
problems. To break these problems, one must devise a reliable 
learning neural network technique that is able to learn under 
a limited weight space in milliseconds and a novel architec- 
ture that is fully programmable through instruction sets, from 
which the real time adaptive network in a chip can be achieved 
to solve a real time applications in a dynamic environment. 

Hence, there is a need in the art for a fully programmable 
processing architecture that can address complex data fusion 
applications in a dynamic environment. 

SUMMARY 

Embodiments of the present invention comprise a system 
and method for cognitive processing of sensor data. A pro- 
cessor array receiving analog sensor data and having pro- 
grammable interconnects, multiplication weights, and filters 
may provide for adaptive learning in real-time. A static ran- 
dom access memory may contain the programmable data for 
the processor array and the stored data is modified to provide 
for adaptive learning. 

One embodiment of the present invention is a system for 
processing sensor data comprising: an input/output bus, 
wherein the input/output bus receives sensor data and outputs 
processed sensor data; a processor array, wherein the proces- 
sor array receives analog signals containing sensor data from 
the input/output bus and the processor array comprises one or 
more matrices of analog multiplication nodes and transfer 
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function elements and the processor array outputs processed 
sensor data from the one or more matrices of analog multi- 
plication nodes and transfer function elements ; and a memory 
containing data values controlling configurations of the pro- 
cessor array. 

Another embodiment of the present invention is a method 
for processing sensor data comprising: receiving sensor data 
in a plurality of first analog data streams; applying a first set 
of multiplicative weights to the plurality of first analog data 
streams to produce a plurality of first multiplied analog data 
streams; filtering the plurality of first multiplied analog data 
streams based on a first set of filter characteristics to produce 
a plurality of preprocessed analog data streams; providing a 
plurality of second analog data streams based on the plurality 
of preprocessed analog data streams; applying a second set of 
multiplicative weights to the plurality of second analog data 
streams to produce a second plurality of multiplied analog 
data streams; filtering the second plurality of multiplied ana- 
log data streams based on a second set of filter characteristics 
to produce a plurality of processed analog data streams; and 
outputting the plurality of processed analog data streams to 
produce processed sensor data. 

BRIEF DESCRIPTION OF THE SEVERAL 
VIEWS OF THE DRAWINGS 

FIG. 1 shows a system block diagram including a Cogni- 
tive Computing Architecture. 

FIG. 2 shows a Cognitive Computing Architecture. 

FIG. 3 shows an architecture for a Hybrid Intelligent Pro- 
cessor Array. 

FIG. 4 shows a block diagram of a Multiplying Digital-to- 
Analog Converter. 

FIG. 5 shows an electrical schematic of a 10-bit Multiply- 
ing Digital-to -Analog Converter. 

FIGS. 6A-6C show the Elaine image and results from 
reconstructing the Elaine image. 

FIG. 6D shows the results of a 10 component vector 
extracted from the Elaine image shown in FIG. 6A. 

FIGS. 7A-7C show a tank image and results from recon- 
structing the tank image. 

FIG. 7D shows the results of a 10 component vector 
extracted from the tank image shown in FIG. 7A. 

FIG. 8A shows an image of the MARS yard at the Jet 
Propulsion Lab at 3 PM. 

FIG. 8B shows the segmented output of the image shown in 
FIG. 8A. 

FIGS. 9A-9C show the 4 PM scene of FIG. 8 A and the 
segmented results of that scene. 

FIGS. 10A-10C show the 5 PM scene of FIG. 8A and the 
segmented results of that scene. 

FIGS. 11A-19B show various views of a scene and an 
object to be detected and tracked within the various views of 
that scene. 

FIGS. 20A-20C show the mixing of odorants within an 
environment, the detection of those odorants, and the actual 
odorant composition of the environment. 

DETAILED DESCRIPTION 

A system block diagram comprising an embodiment of the 
present invention having a Cognitive Computing Architec- 
ture is shown in FIG. 1. As shown on FIG. 1, the system 100 
comprises an input block 110, an output block 120, a data bus 
block 130, a processing block 140 and a storage block 150. 
The input block 110 may comprise sensing devices such as 
Infrared (IR) sensors, Light Detection and Ranging (LIDAR) 
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sensors, Radio Detection and Ranging (RADAR) sensors, 
visuals sensors, chemical sensors, bio -sensors, olfactory sen- 
sors, and any other such sensors. The output block 120 may 
comprise devices that provide output signals to other receiv- 
5 ing elements. The output signals may include, but are not 
limited to, visual indicators, electrical signals, mechanical 
actuation signals, radio -frequency signals. The other receiv- 
ing elements may comprise elements such as machines, 
humans, or computing devices. The processing block 140 
10 may be configured to process fully parallel analog data from 
the input block 110 via the data bus block 130 to allow real 
time processing. The processing block 140 may comprise 
processing techniques such as Principal Component Analysis 
(PCA), Independent Component Analysis (ICA), Neural Net- 
15 work (NN), Genetic Algorithm (GA), or other techniques. In 
a preferred embodiment of the present invention, the process- 
ing block 140 is capable of reconfiguration and adaptation as 
required when a target is changing in a dynamic environment. 
Moreover, the processing block 140 or portions of the pro- 
20 cessing block 140 may be configured to a particular architec- 
ture as determined by processing requirements, e.g. prepro- 
cessing, processing, static or adaptive requirements. The 
storage block 150 may store knowledge of data collected, 
instead of raw or information data as typically done with 
25 current computing architectures, e.g. digital machine. When 
required, knowledge data in the storage block 150 can be 
reconstructed to obtain the raw data. This capability may be 
considered as a cognitive approach. 

An embodiment of the present invention comprises a Cog- 
30 nitive Computing Architecture 200 which includes a Digital/ 
Analog I/O Bus 210, Hybrid Intelligent Processor Array 220, 
and Static Random Access Memory (SRAM) 230 is shown in 
FIG. 2. These elements are described in additional detail 
below. In the architecture depicted in FIG. 2, the I/O Bus 210 
35 may be configured to receive digital or analog data, but the 
Processor Array 220 is configured to operate on analog data. 
Therefore a digital to analog converter 250 may be used to 
convert digital input data into analog data for processing by 
the Processor Array 220 or other portions of the architecture. 
40 The architecture may also include an analog to knowledge 
element 240 that provides the ability to extract knowledge 
features (in a digital representation) from an analog stream 
for storage in or access by the SRAM 230. 

As shown in FIG. 2, the I/O bus 210 is able to accommodate 
45 both analog and digital signals. With this hybrid I/O scheme, 
the Processor Array 220 can reduce I/O bottleneck and per- 
form fully parallel and high speed computation for adaptive 
learning and execution operations, while maintaining a digi- 
tal data format for instruction sets and network reconfigura- 
50 tion. Moreover, the hybrid I/O will be a natural retrofit to 
multiple sensory data types (e.g., IR, visual data etc.). The I/O 
bus may accommodate several streams of analog input data. 
In a preferred embodiment, the input data bus may comprise 
128 analog data lines, which, if a 10 bit digital resolution were 
55 assumed, would comprise a 1280 bit (128x10 bits) data bus. 

The SRAM 230 may be to store the instruction codes, 
system configurations, and/or digital data sets. It may also 
reconfigure the network within the Processor Array 220 so 
that it can be programmed to a particular set of processors 
60 suitable for a particular application using a reconfigurable 
computing technique (such as Field Programmable Array, 
System on a Chip, network processor, etc.). The SRAM may 
also store feature data, not raw data as in traditional 
approaches. Accesses to the SRAM would then allow recon- 
65 struction of the raw information if needed. The form of fea- 
ture data is the knowledge as selected information and 
enables the readiness of use as cognitive processing. 
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The Hybrid Intelligent Processor Array 220 is an adaptive 
processor comprising a matrix of, preferably, 10-bit connec- 
tion strength (weight) and an array of processing units. 
Depending on the data type(s) from one or multiple sources, 
one or several data filters can be reconfigured to perform 
adaptive learning in real time. The data rates of different types 
of sensory data do not necessarily have to be the same. To 
perform the real time adaptive learning, the instruction set is 
stored in SRAM and may run as fast as the raw data rate in the 
sensor I/O channels. The Processor Array 220 configuration 
allows for high speed, fully parallel, and low power process- 
ing since sensory input signals are in analog, fully parallel and 
asynchronous. 

FIG. 3 shows an architecture for a Hybrid Intelligent Pro- 
cessor Array 220 according to an embodiment of the present 
invention. In this architecture, there are two blocks, A-block 
310, andB-block320. The A-block 310 comprises a program- 
mable switch array 311, multiplication nodes 313, and trans- 
fer function elements 315. The A-block 310 receives data 
from the I/O bus that i s routed through the switch array 3 1 1 to 
the multiplication nodes 313. The multiplication factors of 
each multiplication node 313 are defined by matrix W x . The 
results are routed to transfer function elements 315 imple- 
menting a transfer function F(.). In a preferred embodiment of 
the present invention, the transfer function elements 315 com- 
prise programmable analog filters. The transfer function F(.) 
may comprise linear, sigmoidal, Gaussian functions or other 
transfer functions. For example, to convert a signal in the time 
domain to a frequency domain such as Discrete Cosine Trans- 
form (DCT), a particular set of coefficient values are down- 
loaded to the W x array, and the multiplication between the 
weight matrix and the input vector will result in a conversion 
with the output transfer function F(.) as a linear function. 
Moreover, the output from F(.) can be fedbackto the input via 
a set of switches programmed in the SRAM 230. This feature 
allows users to be able to manipulate block-A 310 in a high 
order nonlinear network as needed. For example, block-A 
may be configured as cascading neural network architecture 
(see, for example, S. E. Fahlman and C. Lebiere, “The Cas- 
cade Correlation learning architecture,” Advances in Neural 
Information Processing Systems II, Ed: D. Touretzky, Mor- 
gan Kaufmann, SanMateo, Calif., pp. 524-532, 1990 or 19. T. 
A. Duong and Allen R. Stubberud, “Convergence Analysis Of 
Cascade Error Projection — An Efficient Learning Algorithm 
For Hardware Implementation,” International Journal of 
Neural System, Vol. 10, No. 3, pp. 199-210, June 2000) or a 
multi layered neural network (see, for example, B. Widrow, 
“Generalization and information storage in networks of 
ADALINE neurons,” Ed: G. T. Yovitt, “Self-Organizing Sys- 
tems,” Spartan Books, Washington D.C., 1962) 

The B -block 320 is cascaded with the A-block 3 1 0 in which 
the output of the A-block 310 can be inputted into the B-block 
320. The array of the B-block 320 is similar to the A-block 
310, where the B-block 320 comprises a programmable 
switch array 321, multiplication nodes 323, and transfer func- 
tion elements 325. In a preferred embodiment of the present 
invention, the transfer function elements 325 comprise pro- 
grammable analog filters. However, the B-block 320 is pref- 
erably implemented with a cascading architecture array 
rather than the squared array of the A-block 310. The transfer 
function G(.) is also a programmable function. The output of 
the B-block 320 can be programmed to feedback to the input 
of the B-block 320 and/or the input of the A-block 310. The 
programmable switches 311, 321 allow the reconfiguration of 
the Processor Array into a more complicated network as 
needed. 
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From block-A and block-B, each switch can be pro- 
grammed to become a multi-layered neural network (MLP), 
cascading neural network (CNN), principle component 
analysis (PCA), independent component analysis, least 
5 mean-squared network (LMS), DCT, FFT, etc. 

The multiplication nodes 313, 323 may be implemented as 
1 0-bit Multiplying Digital to Analog Converters (MDACs) . A 
block diagram of an exemplary 10-bit MDAC 400 is shown in 
FIG. 4. The MDAC 400 implements a multiplier 401 between 
to analog input current and a 10-bit digital weight D stored in 
SRAM 407, and the output of the MDAC is in a current mode. 
Since the MDAC operates in a current mode, multiple sources 
may provide current to the MDAC 400 and the MDAC 400 
may provide current to multiple devices. Such a capability 
15 supports the multiple node arrays discussed above. Switches 
403, 405 allow the reconfiguration of the MDAC 400 to load 
different multiplication factors and can be controlled from a 
part of SRAM block. This allows the system architecture to be 
flexible to achieve various tasks. The 10-bit digital weight of 
20 the MDAC 400 is provided by the elements of the and W 2 

matrices. This MDAC implementation provides that the pro- 
cessing array may be compact, consume low power, have a 
relatively simple architecture, and exhibit parallelism. 

FIG. 5 shows an electrical schematic of an exemplary 
25 MDAC 500. In this design, the coefficient can be digi- 
tized and stored into SRAM (D0-D9) and the digital signal 
input is converted into current l in and 32*I m for optimizing the 
design space. The multiplier is used to multiply W k and \ in and 
its result in the current mode is \ out . In FIG. 5, the dotted block 
30 510 is an SRAM design, which will take Data. w to D. when 
Load is high and store D z when Load is low (and Load^ is 
high). 

As digital W k is written into the SRAM array and \ in is 
available, multiplication is accomplished by conditionally 
35 scaling the input current \ in by a series of current mirrored 
transistors 511. For each current mirror, a pass transistor 513 
controlled by one bit from the SRAM which conditionally 
allows the current to be placed on a common summation line. 
The bits in the digital word from LSB to MSB are connected 
40 to 1, 2, 4 . . . and 512 current mirror transistors, so that the 
input current is scaled by the appropriate amount. To optimize 
the number of current mirror transistors (the total is 1023 
current minor transistors) for space and speed due to gate 
capacities of minor transistors, the 10-bit multiply is split into 
45 two 5-bit multiplies (two of 3 1 current mirror transistors) and 
each of them (two) is taken through different input currents: 
\ in or 32*I IW . The resulting summation current is unipolar. 
However, a current steering differential transistor pair 521, 
controlled by a tenth bit of the digital word, determines the 
50 direction of the current output, such that two -quadrant mul- 
tiplication is accomplished (-1024 to 1024 levels). 

In embodiments according to the present invention, data 
can come directly from sensors in an analog form. Using an 
analog I/O bus (such as that shown in FIG. 2), the I/O bus can 
55 digest information flow as fast as the sampling rate of the 
sensors. However, the raw data may expose some difficulties 
for reliable processing. One technique often used is a prepro- 
cessing step and this step can vary from one type of data to 
another. For example, the principle component analysis can 
60 be a useful tool for visual data by reducing the background, 
while still maintaining the principle information for process- 
ing. In such a technique, block-A 310 as shown in FIG. 3 may 
be used to implement preprocessing. In addition, the indepen- 
dent component analysis can be useful for audio sensory data 
65 to reduce cross talk and background noise. When the prepro- 
cessed filter is reconfigured and ready, a real time adaptive 
learning for preprocessing will take place to adapt needed 
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information. When finished, processing units (which may be 
implemented in block-B 320 as shown in FIG. 3 ) can be 
reconfigured accordingly, and the learning of a suitable filter 
to solve the problem will be accomplished. The learning 
iterations and reconfiguration system can be coded as 
machine (i.e., instruction) code to provide instructional steps 
for a particular application. This machine will act as switch- 
ing control to activate the information flow path for adaptive 
learning step. Further, multiple instantiations of the architec- 
ture shown in FIG. 3 may be cascaded into a larger system as 
needed for other applications. 

A feature of embodiments of the present invention is the 
ability to dynamically reconfigure the architecture of the Pro- 
cessor Array. That is, in some embodiments, the data received 
by the Processor Array may result in the configuration of the 
switch array, weights used by the multiplication nodes, or the 
transfer functions of the transfer function elements being 
changed in real-time. Hence, the Processor Array may be 
flexibly configured to implement learning and be adaptive to 
the data received and the processing to be provided. This 
flexibility is typically not present in other digitally -based 
hardware platforms, such as Field Programmable Gate 
Arrays, Digital Signal Processors, microprocessors, etc. 

As indicated above, the Processing Array may be consid- 
ered as providing preprocessing with block-A and processing 
with block-B. The discussion below presents results that may 
be obtained in using the Processing Array in various applica- 
tions. 

A first application is the use of block-A as a real time 
adaptive PC A filter for feature extraction and data compres- 
sion. Two gray scale images were used: Elaine (as shown in 
FIG. 6 A and tank (as shown in FIG. 7A). The Elaine image 
consisted of 256x256 gray pixels and each pixel had 8-bit 
quantization. The tank was a 512x512 pixel image with 8-bit 
gray scale/pixel. 

An input vector as row data with 64 pixel s/vector was used 
to construct the training vector set. When the training vector 
set is available, the algorithm as shown in EQ. 1 below is 
applied to extract the principal vector. Analysis showed that 
the maximum number of iterations required was 1 50 of learn- 
ing repetitions and the first 20 component vectors are 
extracted. Feature extraction using PCA is a well known 
approach (see, for example, T. A. Duong and Allen R. Stub- 
berud, “Convergence Analysis Of Cascade Error Projec- 
tion — An Efficient Learning Algorithm For Hardware Imple- 
mentation,” International Journal of Neural System, Vol. 10, 
No. 3, pp. 199-210, June 2000) and is based on the most 
expressive features (eigen vectors with the largest eigenval- 
ues). 

The first 10-component vector extracted from Elaine 
image using the block-A architecture is projected onto the 
first 10-component from MATLAB (inner product) and its 
results are shown in FIG. 6D. The first 10-component vector 
extracted from tank image using MATLAB and the block-A 
architecture and the projection between principal vectors are 
shown in FIG. 7D. As orthogonal characteristics between 
principal vectors, if the learning component vector and the 
component vector from MATLAB are the same order and 
identical (or close to identical), the expected inner product 
should be close to +/-1 ; otherwise, it should be close to zero. 
As shown in FIG. 6D and FIG. 7D, there are ten values (+/-1 ) 
and the rest (70 values are close to zero) from which it may be 
concluded that the block-A architecture can extract the fea- 
ture vector with the same results as the MATLAB technique. 

A comparison of feature extraction and image reconstruc- 
tion between a known MATLAB technique and use of the 
block-A configured for PCA was also performed. In this case, 
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the first 20-component vector from full set of 64 component 
vectors was extracted. The full image was then constructed 
from the extracted first 20 component principal vector. FIG. 
6B shows the Elaine image and FIG. 7B shows the Tank 
5 image constructed using the MATLAB technique. FIG. 6C 
shows the Elaine image and FIG. 7C shows the Tank image 
constructed using the block-A configured for PCA. These 
images show that the block-A implementation provides 
results similar to known techniques, but, given the high speed 
to nature of the architecture, at a higher speed. 

Another application is to configure block-B for color seg- 
mentation. A challenging aspect in color segmentation is 
when the light intensity and resolution are dynamically 
changing. It is easily recognized that the initial knowledge 
15 used to train the network will have very little effect at a new 
location and therefore will need to be updated through learn- 
ing of newly extracted data. 

Color segmentation may be applied to a spacecraft guid- 
ance application. In this case, the adaptation process that can 
20 aid in spacecraft guidance may be described as follows: when 
the network that has acquired current knowledge at time t 0 is 
used to test the subsequent image at time t 0 +At, segmentation 
results from the image at t 0 +At will be used to extract the 
training set to update the previous knowledge of the network 
25 at time t 0 . This process of repeatedly segmenting and updat- 
ing is performed until a spacecraft attempting a landing on a 
surface reaches its destination. 

While the process of segmenting and updating are desired 
characteristics of an adaptive processor, one issue is how 
30 often such updates are necessary. The frequency of updates 
has a direct impact on power consumption. More power is 
consumed if updates are performed between each sequential 
image. The problem with infrequent updates, however, is that 
the network may not interpolate easily based upon new 
35 images from which the newly segmented data may be insuf- 
ficient for training. To find the optimal sampling rate in a 
landing application, At should be “sufficiently small” and will 
depend upon the landing velocity and other environmental 
changes. The sampling rate becomes significant in the actual 
40 design and development of a spacecraft landing system. 

To judge the applicability of the Hybrid Processor Array 
described above to a color segmentation environment that 
may be presented in a spacecraft landing environment, a 
simulation study was conducted based upon images syntheti- 
45 cally derived at JPL. A digital camera was used to obtain a set 
of images at different times of the day, namely, at 3:00 PM, 
4:00 PM, and 5:00 PM, and at different locations within the 
MARS YARD at the Jet Propulsion Laboratory. These images 
were used to validate the neural network performance with 
50 and without adaptive capabilities. In addition, images were 
acquired at 15 second intervals to capture effects of light 
intensity, contrast, and resolution for adaptive learning step. 

To classify each pixel, a pixel to be classified and its imme- 
diate neighbors were used to form a 3x3 sub- window as the 
55 input training pattern (thus each input pattern has 27 elements 
from 3x3 of RGB pixel). Based on a previous study, the 3x3 
RGB input pattern was found to be the optimal size in com- 
parison to using a single RGB input, a 5x5 RGB sub -window, 
or a 7x7 RGB sub-window. In this study, the objective was to 
60 segment the image into three groups: “Rockl ”, “Rock2”, and 
“Sand”. The topology of the studied network was a 27x5x3 
cascading architecture neural network, having 27 inputs, 5 
cascaded hidden units, and 3 output units. 

FIG. 8A shows a 3:00 PM image. From this image, 408 
65 patterns for training data, 588 patterns for cross validation, 
and 1200 patterns for testing were sampled and collected. 
With these sample sets, the learning was completed with 91% 
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correct in training, 90% correct in validation, and 91% correct 
in testing. After training was performed, the segmented out- 
put of the original image of FIG. 8 A is shown in FIG. 8B. 

With the knowledge acquired from FIG. 8A, the network 
was tested with the image input shown in FIG. 9A, which was 
collected at 4:00 PM. The output result is shown in FIG. 9B 
(no intermediate adaptive step was performed). FIG. 9C is the 
output result with the network acquired from the intermediate 
knowledge through adaptive learning. 

In a similar manner, the original image shown in FIG. 10A 
was collected at 5 PM. FIG. 10B is the segmented image with 
the previous training set at 4 PM and FIG. 10C is the seg- 
mented image with an intermediate adaptive step. 

Based on the aforementioned results, it may be concluded 
that the adaptive technique is needed to obtain better segmen- 
tation when the enviromnent is changing rapidly. For a 
MARS landing application, the lander would be moving with 
a velocity towards its landing surface. This would require an 
increased frequency of updates as the lander approaches the 
landing site. As indicated, embodiments of the present inven- 
tion may support the adaptive technique that provides for the 
increased frequency of updates for the MARS lander appli- 
cation. 

Still another application of an embodiment of the present 
invention is dynamic detection and tracking. Such an appli- 
cation may be based on the adaptive shape feature detection 
technique as U.S. patent application Ser. No. 1 1/498,53 1 filed 
Aug. 1, 2006 titled “Artificial Intelligence Systems for Iden- 
tifying Objects.” Such a technique may be considered as more 
robust than local tracking approaches, e.g., particular local 
pixel group. 

FIG. 11B shows an object present in FIG. 11 A that is to be 
detected and tracked in subsequent images. Given the initial 
image in FIG. 11D, the shape feature of the object may be 
processed using in the preprocessing block (i.e., Block A) of 
the Hybrid Processor Array discussed above for detection and 
tracking. When processing of the images of FIG. 11A and 
FIG. 11B is completed, the result is tested in the scene of 
shown in FIG. 12 A and autonomously found the similar 
object shown in FIG. 12B. The object shown in FIG. 12B was 
adapted for a current dynamic and used to test FIG. 13 A and 
autonomously found the similar object shown in FIG. 13B 
from the object shown in FIG. 12B. Remaining FIGS. 14A 
through 19B show the detection and tracking of the objects 
shown in FIGS. 14B, 15B, 16B, 17B, 18B, and 19B from the 
scenes shown in FIGS. 14A, 15A, 16A, 17A, 18A, and 19A. 
From these figures, it can be seen that the Hybrid Processor 
Array may successfully detect the moving car in the dynamic 
scene. 

In another application of an embodiment of the present 
invention, analysis of sensor responses may be employed for 
the detection of chemical compounds in an open and chang- 
ing environment, such as a building or a geographical area 
where air exchange is not controlled and limited. To search 
for a chemical compound to determine if it exists in the 
operating environment, a Spatial Invariant Independent Com- 
ponent Analysis (SPICA), such as that described in T. A. 
Duong, M. Ryan and V. A. Duong, “Smart ENose for Detec- 
tion of Selective Chemicals in an Unknown Environment.” 
Special issue of Journal of Advanced Computational Intelli- 
gence and Intelligent Informatics, Vol. 11, No. 10, pp. 1 197- 
1203, 07, may be used to separate and detect the mixtures in 
the open environment. 

SPICA can be embedded in an embodiment of the Hybrid 
Processor Array described above using both Blocks A and B. 
FIG. 20 A shows the mixing of two unknown odorants at 
unknown concentrations. FIG. 2 OB shows the odorants 
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detected by the SPICA processing performed by an embodi- 
ment of the present invention. FIG. 20C shows the actual 
odorant composition of these mixtures. Hence, it may be 
concluded that an embodiment of the present invention may 
5 be used for detecting chemicals in an unknown environment. 

No limitation is intended by the description of exemplary 
embodiments which may have included tolerances, feature 
dimensions, specific operating conditions, engineering speci- 
fications, or the like, and which may vary between implemen- 
l o tations or with changes to the state of the art, and no limitation 
should be implied therefrom. In particular it is to be under- 
stood that the disclosures are not limited to particular com- 
positions or biological systems, which can, of course, vary. 
This disclosure has been made with respect to the current state 
15 of the art, but also contemplates advancements and that adap - 
tations in the future may take into consideration of those 
advancements, namely in accordance with the then current 
state of the art. It is intended that the scope of the invention be 
defined by the Claims as written and equivalents as appli- 
20 cable. It is also to be understood that the terminology used 
herein is for the purpose of describing particular embodi- 
ments only, and is not intended to be limiting. Reference to a 
claim element in the singular is not intended to mean “one and 
only one” unless explicitly so stated. As used in this specifi- 
25 cation and the appended claims, the singular forms “a,” “an,” 
and “the” include plural referents unless the content clearly 
dictates otherwise. The term “several” includes two or more 
referents unless the content clearly dictates otherwise. Unless 
defined otherwise, all technical and scientific terms used 
30 herein have the same meaning as commonly understood by 
one of ordinary skill in the art to which the disclosure pertains . 

Moreover, no element, component, nor method or process 
step in this disclosure is intended to be dedicated to the public 
regardless of whether the element, component, or step is 
35 explicitly recited in the Claims . No claim element herein is to 
be construed under the provisions of 35 U.S.C. Sec. 1 12, sixth 
paragraph, unless the element is expressly recited using the 
phrase “means for ...” and no method or process step herein 
is to be construed under those provisions unless the step, or 
40 steps, are expressly recited using the phrase “comprising 
step(s) for ...” 

A number of embodiments of the disclosure have been 
described. Nevertheless, it will be understood that various 
modifications may be made without departing from the spirit 
45 and scope of the present disclosure. Accordingly, other 
embodiments are within the scope of the following claims. 

What is claimed is: 

1. A system for processing sensor data comprising: 

an input/output bus, wherein the input/output bus receives 
50 sensor data and outputs processed sensor data; 

a processor array, wherein the processor array receives 
analog signals containing sensor data from the input/ 
output bus and the processor array comprises one or 
more matrices of analog multiplication nodes and trans- 
55 fer function elements and the processor array outputs the 

processed sensor data from the one or more matrices of 
analog multiplication nodes and transfer function ele- 
ments; and 

a memory containing data values controlling configura- 
60 tions of the processor array, 

wherein the processor array comprises: 
a first array block, the first array block comprising: 
a first programmable switch array configured to 
receive the sensor data; 

65 a square array of first array analog multiplication 

nodes, wherein each first array multiplication node 
selectively receives analog signals from the first 
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programmable switch array, or one or more first 
array multiplication nodes, or the first program- 
mable switch array and one or more first array 
multiplication nodes; and 

one or more first array transfer function elements, 
wherein each first array transfer function elements 
receives analog signals from one of the first array 
analog multiplication nodes, and 
a second array block, the second array block comprising: 
a second programmable switch array configured to 
receive signals from the one or more first array 
transfer function elements; 
a cascaded array of second array analog multiplica- 
tion nodes; 

one or more second array transfer function elements, 
wherein each second array transfer function ele- 
ment receives analog signals from one of the sec- 
ond array analog multiplication nodes; 
wherein each second array multiplication node selec- 
tively receives analog signals from the second pro- 
grammable switch array, one or more second array 
multiplication nodes, the second programmable 
switch array and one or more second array multi- 
plication nodes, or one of the one or more second 
array transfer function elements. 

2. The system according to claim 1, wherein outputs from 
one or more of the first array transfer function elements are 
selectively coupled to the first programmable switch array as 
inputs to the first programmable switch array. 

3. The system according to claim 2, wherein outputs from 
one or more of the second array transfer function elements, or 
outputs from one or more of the second array transfer function 
elements, or outputs from one or more of the second array 
transfer function elements and outputs from one or more of 
the second array transfer function elements are selectively 
coupled to the second programmable switch array as inputs to 
the second programmable switch array and selectively 
coupled to the first programmable switch array as inputs to the 
first programmable switch array. 

4. The system according to claim 1, wherein outputs from 
one or more of the second array transfer function elements, or 
outputs from one or more of the second array transfer function 
elements, or outputs from one or more of the second array 
transfer function elements and outputs from one or more of 
the second array transfer function elements are selectively 
coupled to the second programmable switch array as inputs to 
the second programmable switch array. 

5. The system according to claim 1, wherein the memory 
contains data values for selectively controlling the first pro- 
grammable switch array or the second programmable switch 
array or the first programmable switch array and the second 
programmable switch array. 

6. The system according to claim 1, wherein the memory 
contains data values for controlling multiplication factors for 
one or more of the first array multiplication nodes and second 
array multiplication nodes. 

7. The system according to claim 6, wherein one or more of 
the first array multiplication nodes and second array multi- 
plication nodes comprise a current mode multiplying digital 
to analog converter having a digital multiplication factor 
stored in the memory. 

8. The system according to claim 1, wherein data values in 
the memory are modified based upon the processed sensor 
data. 

9. The system according to claim 1, wherein the transfer 
function elements comprise programmable analog filters. 
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10. A method for processing sensor data comprising: 
receiving sensor data in a plurality of first analog data 

streams; 

applying a first set of multiplicative weights to the plurality 
5 of first analog data streams to produce a plurality of first 
multiplied analog data streams; 
filtering the plurality of first multiplied analog data streams 
based on a first set of filter characteristics to produce a 
plurality of preprocessed analog data streams; 

10 providing a plurality of second analog data streams based 
on the plurality of preprocessed analog data streams; 
applying a second set of multiplicative weights to the plu- 
rality of second analog data streams to produce a second 
15 plurality of multiplied analog data streams; 

filtering the second plurality of multiplied analog data 
streams based on a second set of filter characteristics to 
produce a plurality of processed analog data streams; 
and 

20 outputting the plurality of processed analog data streams to 

produce processed sensor data, 
wherein the plurality of first analog data streams com- 
prises: sensor data, sensor data and the plurality of pre- 
processed analog data streams; sensor data and the plu- 
25 rality of processed data streams; or sensor data and the 
plurality of preprocessed analog data streams and the 
plurality of processed data streams, 
wherein the plurality of second analog data streams com- 
prises: the plurality of preprocessed analog data streams, 
30 or the plurality of preprocessed analog data streams and 

the plurality of processed analog data streams, and 
wherein applying a first set of multiplicative weights com- 
prises selectively directing the plurality of analog 
streams to a plurality of first multiplication nodes, 
35 wherein each first multiplication node multiplies a first 
multiplication node input by a weight selected from the 
first set of multiplicative weights to produce a first mul- 
tiplication node output, wherein the first multiplication 
node input comprises: one analog data steam of the 
40 plurality of analog data streams; a first multiplication 
node output from another first multiplication node; or a 
sum of one analog data steam of the plurality of analog 
data streams and a first multiplication node output from 
another first multiplication node, and wherein first mul- 
45 tiplication node outputs from selected first multiplica- 

tion nodes comprise the first plurality of multiplied ana- 
log data streams. 

11. A method for processing sensor data comprising: 
receiving sensor data in a plurality of first analog data 

50 streams; 

applying a first set of multiplicative weights to the plurality 
of first analog data streams to produce a plurality of first 
multiplied analog data streams; 
filtering the plurality of first multiplied analog data streams 
55 based on a first set of filter characteristics to produce a 
plurality of preprocessed analog data streams; 
providing a plurality of second analog data streams based 
on the plurality of preprocessed analog data streams; 
applying a second set of multiplicative weights to the plu- 
60 rality of second analog data streams to produce a second 
plurality of multiplied analog data streams; 
filtering the second plurality of multiplied analog data 
streams based on a second set of filter characteristics to 
produce a plurality of processed analog data streams; 
65 and 

outputting the plurality of processed analog data streams to 
produce processed sensor data, 
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wherein the plurality of first analog data streams com- 
prises: sensor data, sensor data and the plurality of pre- 
processed analog data streams; sensor data and the plu- 
rality of processed data streams; or sensor data and the 
plurality of preprocessed analog data streams and the 5 
plurality of processed data streams, 
wherein the plurality of second analog data streams com- 
prises: the plurality of preprocessed analog data streams, 
or the plurality of preprocessed analog data streams and 
the plurality of processed analog data streams, and to 

wherein filtering the first plurality of multiplied analog 
data streams comprises: 

providing an array of first analog filters, each first analog 
filter having a first analog filter input and a first analog 
filter output and a first analog filter characteristic 15 
selected from the selected first filter characteristics; and, 
directing each multiplied analog data stream of the first 
plurality of multiplied analog data streams into a first 
analog filter input of a selected first analog filter, 
wherein first analog filter outputs comprise the plurality of 20 
preprocessed analog data streams. 

12. A method for processing sensor data comprising: 
receiving sensor data in a plurality of first analog data 
streams; 

applying a first set of multiplicative weights to the plurality 25 
of first analog data streams to produce a plurality of first 
multiplied analog data streams; 
filtering the plurality of first multiplied analog data streams 
based on a first set of filter characteristics to produce a 
plurality of preprocessed analog data streams; 30 

providing a plurality of second analog data streams based 
on the plurality of preprocessed analog data streams; 
applying a second set of multiplicative weights to the plu- 
rality of second analog data streams to produce a second 
plurality of multiplied analog data streams; 35 

filtering the second plurality of multiplied analog data 
streams based on a second set of filter characteristics to 
produce a plurality of processed analog data streams; 
and 

outputting the plurality of processed analog data streams to 40 
produce processed sensor data 
wherein the plurality of first analog data streams com- 
prises: sensor data, sensor data and the plurality of pre- 
processed analog data streams; sensor data and the plu- 
rality of processed data streams; or sensor data and the 45 
plurality of preprocessed analog data streams and the 
plurality of processed data streams, 
wherein the plurality of second analog data streams com- 
prises: the plurality of preprocessed analog data streams, 
or the plurality of preprocessed analog data streams and 50 
the plurality of processed analog data streams, and 
wherein applying a second set of multiplicative weights 
comprises: 

selectively directing the plurality of preprocessed analog 
data streams to a plurality of second multiplication 55 
nodes, wherein each second multiplication node multi- 
plies a second multiplication node input by a weight 
selected from the second set of multiplicative weights to 
produce a second multiplication node output, wherein 
the second multiplication node input comprises: one 60 
preprocessed analog data steam of the plurality of pre- 
processed analog data streams; a second multiplication 
node output from another second multiplication node; or 
a sum of one preprocessed analog data steam of the 
plurality of preprocessed analog data streams and a sec- 65 
ond multiplication node output from another second 
multiplication node, and wherein second multiplication 
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node outputs from selected second multiplication nodes 
comprise the second plurality of multiplied analog data 
streams. 

13. A method for processing sensor data comprising: 
receiving sensor data in a plurality of first analog data 

streams; 

applying a first set of multiplicative weights to the plurality 
of first analog data streams to produce a plurality of first 
multiplied analog data streams; 
filtering the plurality of first multiplied analog data streams 
based on a first set of filter characteristics to produce a 
plurality of preprocessed analog data streams; 
providing a plurality of second analog data streams based 
on the plurality of preprocessed analog data streams; 
applying a second set of multiplicative weights to the plu- 
rality of second analog data streams to produce a second 
plurality of multiplied analog data streams; 
filtering the second plurality of multiplied analog data 
streams based on a second set of filter characteristics to 
produce a plurality of processed analog data streams; 
and 

outputting the plurality of processed analog data streams to 
produce processed sensor data 
wherein the plurality of first analog data streams com- 
prises: sensor data, sensor data and the plurality of pre- 
processed analog data streams; sensor data and the plu- 
rality of processed data streams; or sensor data and the 
plurality of preprocessed analog data streams and the 
plurality of processed data streams, 
wherein the plurality of second analog data streams com- 
prises: the plurality of preprocessed analog data streams, 
or the plurality of preprocessed analog data streams and 
the plurality of processed analog data streams, and 
wherein filtering the second plurality of multiplied analog 
data streams comprises: 

providing an array of second analog filters, each second 
analog filter having a second analog filter input and a 
second analog filter output and a second analog filter 
characteristic selected from the selected second filter 
characteristics; and, 

selectively directing each multiplied analog data stream of 
the second plurality of multiplied analog data streams to 
a second analog filter input of a selected second analog 
filter or to a selected third multiplication node of a plu- 
rality of third multiplication nodes, wherein each third 
multiplication node multiplies a third multiplication 
node input by a weight selected from a third set of 
multiplicative weights selected from the second filter 
characteristics to produce a third multiplication node 
output, wherein the third multiplication node input com- 
prises: a selected second analog filter output; a third 
multiplication node output from another third multipli- 
cation node; or a sum of a selected second analog filter 
output and a third multiplication node output from 
another third multiplication node; and 
selectively directing each second analog filter output to a 
selected third multiplication node of a plurality of third 
multiplication nodes or to the plurality of processed 
analog data streams. 

14. A method for processing sensor data comprising: 
receiving sensor data in a plurality of first analog data 

streams; 

applying a first set of multiplicative weights to the plurality 
of first analog data streams to produce a plurality of first 
multiplied analog data streams; 
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filtering the plurality of first multiplied analog data streams 
based on a first set of filter characteristics to produce a 
plurality of preprocessed analog data streams; 
providing a plurality of second analog data streams based 
on the plurality of preprocessed analog data streams; 5 
applying a second set of multiplicative weights to the plu- 
rality of second analog data streams to produce a second 
plurality of multiplied analog data streams; 
filtering the second plurality of multiplied analog data 
streams based on a second set of filter characteristics to 10 
produce a plurality of processed analog data streams; 
outputting the plurality of processed analog data streams to 
produce processed sensor data; and 
selecting or modifying at least one of the following sets 
based upon the processed sensor data: the first set of 15 
multiplicative weights; the first set of filter characteris- 
tics; the second set of multiplicative weights; and, the 
second set of filter characteristics. 

15. A method for processing sensor data comprising: 
receiving sensor data in a plurality of first analog data 20 
streams; 

applying a first set of multiplicative weights to the plurality 
of first analog data streams to produce a plurality of first 
multiplied analog data streams; 
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filtering the plurality of first multiplied analog data streams 
based on a first set of filter characteristics to produce a 
plurality of preprocessed analog data streams; 
providing a plurality of second analog data streams based 
on the plurality of preprocessed analog data streams; 
applying a second set of multiplicative weights to the plu- 
rality of second analog data streams to produce a second 
plurality of multiplied analog data streams; 
filtering the second plurality of multiplied analog data 
streams based on a second set of filter characteristics to 
produce a plurality of processed analog data streams; 
and 

outputting the plurality of processed analog data streams to 
produce processed sensor data, wherein the processed 
sensor data provides knowledge about a sensor environ- 
ment and the method further comprises selecting or 
modifying at least one of the following sets based upon 
the sensor environment knowledge: the first set of mul- 
tiplicative weights; the first set of filter characteristics; 
the second set of multiplicative weights; and, the second 
set of filter characteristics. 





